Release QAT example with NLS #3480
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: J. Pablo Muñoz <[email protected]> Co-authored-by: Yuan0320 <[email protected]>
github-actions
bot
added
documentation
Signed-off-by: J. Pablo Muñoz <[email protected]> Co-authored-by: Yuan0320 <[email protected]>
ljaljushkin
requested changes
May 7, 2025
There was a problem hiding this comment.
Thank you for the contribution and very extensive evaluation!
It's great to see an improvement on top of baseline with constant LoRA rank!
On a high level, it looks good for me. Most of the logic is implemented in the sample, changes in NNCF are minimized by extending FQ with LoRA.
I have a few remarks to make it better in terms of integration into NNCF.
One thing that is important for potential customers - total time to get the best checkpoint.
Could you please specify in the readme, how long was tuning and search stage in both cases?
examples/llm_compression/torch/qat_with_lora/NLSDownstreamTasks.md
Outdated
Show resolved
Hide resolved
andreyanufr
reviewed
May 7, 2025
examples/llm_compression/torch/qat_with_lora/NLSDownstreamTasks.md
Outdated
Show resolved
Hide resolved
Signed-off-by: J. Pablo Muñoz <[email protected]> Co-authored-by: Yuan0320 <[email protected]>
github-actions
bot
added
NNCF Common
Signed-off-by: J. Pablo Muñoz <[email protected]>
Signed-off-by: J. Pablo Muñoz <[email protected]>
Signed-off-by: J. Pablo Muñoz <[email protected]> Co-authored-by: Yuan0320 <[email protected]>
ljaljushkin
requested changes
May 9, 2025
examples/llm_compression/torch/qat_with_lora/NLSDownstreamTasks.md
Outdated
Show resolved
Hide resolved
examples/llm_compression/torch/qat_with_lora/NLSDownstreamTasks.md
Outdated
Show resolved
Hide resolved
ljaljushkin
reviewed
May 9, 2025
Signed-off-by: J. Pablo Muñoz <[email protected]> Co-authored-by: Yuan0320 <[email protected]>
Signed-off-by: J. Pablo Muñoz <[email protected]>
Co-authored-by: Lyalyushkin Nikolay <[email protected]>
Co-authored-by: Lyalyushkin Nikolay <[email protected]>
Co-authored-by: Lyalyushkin Nikolay <[email protected]>
jpablomch
force-pushed
the
qat_with_nls_release
branch
5 times, most recently
from
45eb700 to
f8dd856
Compare
Signed-off-by: J. Pablo Muñoz <[email protected]>
jpablomch
force-pushed
the
qat_with_nls_release
branch
from
f8dd856 to
8a5b7db
Compare
Signed-off-by: J. Pablo Muñoz <[email protected]> Co-authored-by: Yuan0320 <[email protected]>
jpablomch
force-pushed
the
qat_with_nls_release
branch
from
b77ba4a to
407fbb3
Compare
github-actions
bot
added
NNCF TF
Signed-off-by: J. Pablo Muñoz <[email protected]> Co-authored-by: Yuan0320 <[email protected]>
jpablomch
force-pushed
the
qat_with_nls_release
branch
from
870e9fe to
28e8f71
Compare
Signed-off-by: J. Pablo Muñoz <[email protected]>
github-actions
bot
removed
NNCF TF
Signed-off-by: J. Pablo Muñoz <[email protected]> Co-authored-by: Yuan0320 <[email protected]>
Signed-off-by: J. Pablo Muñoz <[email protected]> Co-authored-by: Yuan0320 <[email protected]>
ljaljushkin
approved these changes
May 20, 2025
andreyanufr
approved these changes
May 20, 2025
ljaljushkin
reviewed
May 20, 2025
Signed-off-by: J. Pablo Muñoz <[email protected]>
|
Latest job for test examples passed - https://github.com/openvinotoolkit/nncf/actions/runs/15141407340/attempts/1 |
Signed-off-by: J. Pablo Muñoz <[email protected]>
Signed-off-by: J. Pablo Muñoz <[email protected]> Co-authored-by: Yuan0320 <[email protected]>
Co-authored-by: Yuan0320 <[email protected]>
alexsu52
approved these changes
May 21, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Adds example to use NLS fine-tuning with quantization-aware LoRA on downstream tasks.
Reason for changes
To support fine-tuning for downstream scenarios, and NLS often boost the performance of LoRA fine-tuning on downstream tasks.
Related tickets
https://jira.devtools.intel.com/browse/CVS-166802
Tests
See the results in NLSDownstreamTasks.md. We have conducted extensive evaluation on 11 language models and 4 downstream tasks.
examples job: https://github.com/openvinotoolkit/nncf/actions/runs/14934370942